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Remarks/Argument 

Applicant submits this reply to the Office Action of January 27, 2005. Following this 
Amendment, claims 1-24 remain in this application, with ten (10') independent and fourteen (14) 
dependent claims. Original claims 1-23 were rejected under 102(e) as anticipated by U.S. Patent 
No. 6,690,371 ("Okerlund"). Claims 1-23 are substantially unchanged, with only minor edits to 
correct minor typographical or grammatical errors. Nevertheless, for at least the reasons 
discussed below, Applicant respectfully traverses this rejection. 

Applicant traverses the characterization of Okerlund as relevant prior art, given that 
Okerlund does not teach or suggest the present invention. Specifically, as noted in the abstract, 
Okerlund is directed to: 

A method for rapid extraction and visualization of relevant data 
from a volume of image data includes rapidly producing reduced- 
fidelity images derived from an image volume, the reduced fidelity 
images having an adjustable visual parameter; adjusting the visual 
parameter of the reduced-fidelity images during the rapid 
production to select a desired adjustment; producing a full-fidelity 
image derived from the image volume; and applying the selected 
adjustment to the full-fidelity image. 

Thus, Okerlund teaches the visualization of image data, not the capture of an screen image and 
processing of the captured screen image to control the application which generated the image. 
Indeed, no where does Okerlund describe or suggest the capturing of any information displayed 
on a screen and perform decision analysis based on this information capture as provided by the 
present invention, as explained below, fflf 56-58. 

The present invention, is directed to the automated control and execution of programs by 
analyzing the inputs used to generate certain program states, which are ascertained by polling a 
screen capture of a running application program. The present invention seeks to address the 
problem of integrating legacy and newer systems U 6; f 7. This is accomplished with runtime 
agents that use models to intelligently navigate systems, so that a single user interface may be 
used to control multiple applications, f 18. To achieve this, the runtime agents determine the 
state of an application by polling the screen displayed by the application. If 12. Through this 
polling, unique identifiers may be found and noted, thereby informing the integrating program of 
the state of the program. f 12. Alternatively, a programmer may select regions of a screen for 
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the runtime agents. ^ 13. The present invention provides for models that are used by runtime 
agents to intelligently navigate systems. Embodiments of the present invention may employ a 
recorder that "records in the computer memory a plan domain file which comprises each state of 
the presentation layer of another computer system, the available actions from each state, and the 
effect of any actions available in each state though navigating the other computer system in the 
user interface." f 20. Embodiments of the present invention may employ a "fingerprinter" that 
"captures sections of each screen of the presentation space from at least one other computer 
system." 1 28. No where in Okerlund is there any suggestion of a runtime agent acquiring a 
screen capture of a presentation space. Moreover, Okerlund does not teach or suggest the 
simultaneous use of two different processes - one being the runtime agent and the other being the 
application from which the runtime agent captures sections of each screen of the presentation 
space. 

Thus, Okerlund is neither reasonably pertinent to the particular problem with which the 
Applicant was concerned, nor directed to the field of Applicant's endeavor. Okerlund should not 
be deemed prior art. In any event, Okerlund does not disclose or suggest various features of the 
claimed invention and cannot therefore, anticipate claims 1-24. 

The system of claim 1 comprises a "screen fingerprinter" stored in a computer memory 
that "selects at least one region and/or pattern of the screens of the presentation space of a 
computer application to be captured such that said at least one region and/or pattern of each 
screen is unique." Claim 1 (emphasis added). Similarly, the specification notes that a 
fingerprinter "captures sections of each screen of the presentation space from at least one other 
computer system." 1 28. Nowhere does Okerlund teach or suggest the capturing of a region 
and/or pattern of a screen of a presentation space. Examiner's statement that a "a fingerprinter is 
simply the software that acquires specific screen presentation data related to at least one 
computer" is an oversimplification of the meaning of "screen fingerprinter" as understood by one 
of ordinary skill in the art. The fingerprinter, as noted in the claim language and specification 
referenced above, acquires its data from data that is actually displayed upon a screen of a 
presentation space, not merely data that might eventually be displayed or modified to be 
displayed, as is Okerlund. In Okerlund, the end result and objective is to render some 
visualization on a screen. The present invention, in contrast, uses what is already being displayed 
on a screen by another computer application as a data source for runtime agents. This 
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understanding is confirmed by the specification, which notes that a "fingerprinter in the computer 
memory captures sections of each screen of the presentation space from at least one other 
computer system." ^ 28. Thus, the fingerprinter' s source is the actual screen display, which in 
Okerlund, the image on the screen is never used as a source of information. Moreover, Okerlund 
does not disclose the use of one process (performed by "a screen fingerprinter") to observe the 
operation of another process ("a computer application"), by monitoring the other process 5 state 
by selecting "at least one region and/or pattern of the screens of the presentation space of the 
other process. 

Examiner cites col. 2, 11. 22-35 of Okerlund. Here, Okerlund merely describes a system 
for displaying data acquired from an exam prescription subsystem, which acquires its data from a 
medical scan. Okerlund in no way teaches or suggests the use of a screen capture of any image 
displayed as part of an image space. The present invention comprises "a screen finger printer" 
which selects obtains its data from images presented on a screen, see claim 1, Okerlund, in 
contrast, obtains its data from a medical scan. Col. 3, 1. 25-46. Okerlund does in any way teach 
the data acquisition of images displayed on a screen, and indeed never teaches the use of one 
process to observe and select the display of another process. Applicant respectfully requests the 
allowance of claim 1 , and claims 2-4, which are dependent thereon. 

With respect to claim 2, Examiner cites col 3, 11. 35-47 of Okerlund. Here, Okerlund 
does not disclose or suggest the use of a fingerprinter, which is understood as an agent that 
selects a portion of an image already being displayed by another application. Claim 2 is not 
anticipated by Okerlund and should be allowed. 

With respect to claim 3, Examiner cites col. 8, 1. 28-42. Here, Okerlund does not teach or 
discuss the use of a "decision tree." Although a decision tree is one type of hierarchical data 
structure used to make decisions. The data structure noted in Okerlund is merely of data, not 
data that permits the making of decisions. Claim 3 is not anticipated by Okerlund and should be 
allowed. 

With respect to claim 4, Examiner cites col. 3, 11 35-47. For the reasons stated above, 
Okerlund neither teaches the use of a "fingerprinter" nor the use of a "decision tree." Claim 4 is 
not anticipated by Okerlund and should be allowed. 

Claim 5 has been amended to include "a user interface" in order to provide an antecedent 
basis for "said user interface." The system of claim 5, as amended, comprises "a recorder" that 
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records in computer memory "a plan domain file which comprises each screen of the presentation 
layer of a computer application, the keystrokes and/or programs necessary to reach each state, the 
available actions from each state of each screen and the effect of any actions available in each 
state through navigating said computer application in said user interface." Examiner cites Col. 3, 
11. 10-47 and col. 4, 11. 20-40. Neither here, nor elsewhere, does Okerlund teach or suggest the 
use of a "recorder," which by its plain terms indicates something which records a state (which is 
understood by one of ordinary skill, in light of the claims and specification, to be the recording of 
the state of another computer application). Nothing in Okerlund teaches or suggests the use of a 
domain file that comprises ''each screen of the presentation layer of a computer application." 
Nothing in Okerlund teaches or suggests the use of a domain file that comprises "the keystrokes 
necessary to reach each state" Nothing in Okerlund teaches or suggests the use of a domain file 
that comprises "the available actions from each state of each screen and the effect of any 
actions available in each state through navigating said computer application in said user 
interface" Indeed, nothing is Okerlund teaches or suggests the use of an agent to record the 
operations of a computer application by recording what is displayed upon a screen. Claim 5 is 
not anticipated by Okerlund and should be allowed. 

With respect to claim 6, Examiner cites col. 4, 11. 37-42. Here, Okerlund merely discloses 
the viewing of a 3-d visualization. Okerlund does not teach or suggest the generation of a file 
based upon an user's navigation of another computer system. The term "navigation," as 
understood by one of ordinary skill in the art in light of the claims and specification, refers to the 
interaction between a user and another computer application as a user explorers or undertakes use 
of the computer application. Okerlund merely discloses the remote viewing of what is already 
being displayed, not the recording of the activities of a user as the user interacts with another 
computer application. Claim 6 is not anticipated by Okerlund and should be allowed. 

With respect to claim 7, Examiner cites col. 4, 11. 29-36. Here, Okerlund does not teach 
or suggest the automated navigation of another computer application. Claim 7 is not anticipated 
by Okerlund and should be allowed. 

With respect to claim 8, Examiner cites col. 4, 1. 20-40. As noted before, Okerlund does 
not teach or suggest the use of fingerprints nor domain files. Claim 8 is not anticipated by 
Okerlund, which does not even discuss a program's state, and should therefore be allowed. 

With respect to claim 9, Examiner cites col. 4, 11. 54-63. Here, Okerlund does not include 
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pre-conditions or post-conditions for each state. These are not disclosed in Okerlund. Claim 9 is 
not anticipated by Okerlund and should be allowed. 

The system of claim 10 comprises "a computer application model." As understood by 
one of ordinary skill in the art in light of the specification and claims, "a computer generated 
model" is generated by a recorder, thus provides information concerning the various states of an 
application, including the fingerprints of a state, the keystrokes needed to obtain the state, and the 
effect of any actions available in each state through navigating said computer application. % 28; f 
20. These are not disclosed in Okerlund, as cited by the Examiner. Col. 3, 11. 10-21. Claim 10 is 
not anticipated by Okerlund and should be allowed. 

Moreover, the system of claim 10 comprises "a navigation planner." Examiner states that 
a hierarchical data structure is a form of a navigation planner. This is incorrect. By its plain 
terms, a navigation planner plans how a computer application is to be navigated. A mere 
hierarchical data structure does not perform any actions on its own. An navigation planner, in 
contrast, must plan how a computer application is to be navigated to progress from one state to 
another. Nothing in Okerlund, as cited by the Examiner, Col. 8, 11. 29-42, suggests a navigation 
planner, let alone the use of an agent which navigates another computer program. Claim 10 is 
not anticipated by Okerlund and should be allowed. 

With respect to claim 1 1, Examiner cites col. 8, 11. 60-67. Nothing in Okerlund suggests 
the use creation of a new navigation plan, that is, a new way by which a computer application is 
to be navigated or controlled. Claim 1 1 is not anticipated by Okerlund and should be allowed. 

Claim 12 comprises a "recorder," which is synonymous with "recorder." As discussed 
above in connection with claim 5, Okerlund does not teach or suggest a recorder. Claim 12 also 
comprises a "fingerprinter." As discussed above in connection with claim 5, Okerlund does not 
teach or suggest a recorder. Claim 12 also requires the ability to form additional relationships 
between a screen and a domain file can be input through said user interface such that said 
computer application model generator can model said computer application. Examiner cites col. 
3, 11. 36-47. Here, Okerlund does not teach or suggest the modeling of a computer application. 
Moreover, Okerlund does not teach or suggest the use of domain files, as discussed above in 
connection with claim 5. Claim 12 is not anticipated by Okerlund and should be allowed. 

Claim 13 comprises a "runtime agent." Examiner misconstrues the meaning of runtime 
agent as used in the application. A run-time agent: "use the models to intelligently navigate each 
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system" and "can be used to implement systems with a single user interface." Tf 18. Okerlund 
does not teach or suggest the use of runtime agents, which are not merely "involved in the 
execution of code," but must be capable of navigating another computer application. Claim 13 
further comprises a "computer application model" which, as described above in connection with 
claim 10, is not taught or suggested by Okerlund. Claim 13 is not anticipated by Okerlund and 
should be allowed. 

Claim 14 comprises a step of "taking a screen capture of each screen of the presentation 
layer of a computer application." Examiner cites col. 8, 1. 27-67. Here, Okerlund is discusses 
rendering of a three-dimensional image based upon image slices obtain from a medical scan. 
Nothing in Okerlund teaches or suggests taking a "screen capture," that is, the capturing of an 
image of what is displayed on a screen. 

Claim 14 further comprises "selecting areas of said screen captures to be examined for 
the presence of an attribute in said area." Examiner cites col. 8, 11. 7-17. Claim 14 further 
comprises "creating a decision tree such that each of said screen captures has a unique end node 
of said decision tree." As discussed above in connection with claim 3, nothing in Okerlund 
teaches or suggests the use of a decision tree. Claim 14 is not anticipated by Okerlund and 
should be allowed. 

With respect to claims 15 and 16, Examiner cites col. 3, 11. 36-47 and col. 8, 11. 7-17. 
Here, Okerlund does not teach or suggest the use of screen captures, and thus does not anticipate 
the requirement of claim 15 and claim 16 that areas within such screen captures are, respectively, 
selected automatically and manually. Claims 15 and 16 are not anticipated by Okerlund and 
should be allowed. 

With respect to claim 17, Examiner cites col. 3, 11. 36-47; col. 8, 11. 27-47. As discussed 
above in connection with claim 3, Okerlund does not teach or suggest the use of a decision tree. 
Claim 17 is not anticipated by Okerlund and should be allowed. 

Claim 18 requires the "recording in a plan domain file each screen of the presentation 
layer of said computer application, the keystrokes necessary to reach each state of each screen of 
said computer application, the states of each screen, and the effect of any actions taken on each 
screen." As discussed above with respect to claim 5, Okerlund does not teach or suggest any of 
these features. Claim 18 is not anticipated by Okerlund and should be allowed. Moreover, 
claims 19 and 20, which depend from claim 18, should similarly be allowed. 
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Claim 21 requires "accessing at least one computer application model that encapsulates 
information on how at least one computer application is controlled and/or data is accessed." As 
discussed above in connection with claim 10, Okerlund does not teach or suggest the use of a 
computer application model. Claim 21 further comprises a step of "planning a path through said 
at least one computer application that will achieve the goal of said problem statement." As 
discussed above in connection with claim 10, Okerlund does not teach or suggest the use of 
navigation planning. Claim 21 is not anticipated by Okerlund and should be allowed. 

Claim 22 comprises a step of "taking a screen capture of each screen of the presentation 
layer of a computer application." As discussed above in connection with claim 1, Okerlund does 
not teach or suggest the use of a "screen capture." Claim 22 further comprises a step of "creating 
a decision tree such that each of said screen captures has a unique end node of said decision 
tree." As discussed above in connection with claim 3, Okerlund does not teach or suggest the use 
of decision trees. Claim 22 further comprises "recording in a plan domain file each screen of the 
presentation layer of said computer application, the keystrokes necessary to reach each state of 
each screen of said computer application, the states of each screen, and the effect of any actions 
taken on each screen." As discussed above in connection with claim 5, Okerlund does not teach 
or suggest the use of a "domain file" as is understood by one of ordinary skill in the art in light of 
the claims and specification. 

Applicant respectfully traverses examiner's rejection of claims 1-23 as being anticipated 
by Okerlund and requests the allowance of claims 1-23 in view of the arguments above. 
Likewise, it is believed that the present invention similarly overcomes the other cited reference 
for these and other reasons. New claim 24 has been added to clarify the above noted differences 
and should be allowed for the reasons stated above. 
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Conclusion 

In view of the foregoing, Applicant respectfully submits that this application is now in 
condition for allowance. Favorable reconsideration and prompt allowance are earnest solicited. 
Should the Examiner believe that anything further is necessary to expedite prosecution of this 
application, the Examiner is invited to contact Applicant's representative at the telephone 
number listed below. 

If there are any additional fees due in connection with the filing of this response, please 
charge the fees to our Deposit Account No. 50-1349. 



Dated: November 14, 2005 

HOGAN & HARTSON LLP 

555 13 th Street, N.W. 
Washington, D.C. 20004 
Telephone: 202-637-5600 
Facsimile: 202-637-5910 
Customer No. 24633 



Respectfully submitted, 




Celine Jimenez Crowson 
Registration No. 40,357 



David D. Nelson 
Registration No. 47,818 



14 



